Symbolic LAO* Search for Factored Markov Decision Processes
نویسندگان
چکیده
We describe a planning algorithm that integrates two approaches to solving Markov decision processes with large state spaces. It uses state abstraction to avoid evaluating states individually. And it uses forward search from a start state, guided by an admissible heuristic, to avoid evaluating all states. These approaches are combined in a novel way that exploits symbolic model-checking techniques and demonstrates their usefulness in solving decision-theoretic planning prob-
منابع مشابه
Memory-Effcient Symbolic Online Planning for Factored MDPs
Factored Markov Decision Processes (MDP) are a de facto standard for compactly modeling sequential decision making problems with uncertainty. Offline planning based on symbolic operators exploits the factored structure of MDPs, but is memory intensive. We present new memoryefficient symbolic operators for online planning, prove the soundness of the operators, and show convergence of the corresp...
متن کاملSymbolic Heuristic Search Using Decision Diagrams
We show how to use symbolic model-checking techniques in heuristic search algorithms for both deterministic and decision-theoretic planning problems. A symbolic approach exploits state abstraction by using decision diagrams to compactly represent sets of states and operators on sets of states. In earlier work, symbolic model-checking techniques have been used to find plans that minimize the num...
متن کاملSymbolic Heuristic Search Value Iteration for Factored POMDPs
We propose Symbolic heuristic search value iteration (Symbolic HSVI) algorithm, which extends the heuristic search value iteration (HSVI) algorithm in order to handle factored partially observable Markov decision processes (factored POMDPs). The idea is to use algebraic decision diagrams (ADDs) for compactly representing the problem itself and all the relevant intermediate computation results i...
متن کاملSolving Markov Decision Problems Using Heuristic Search
We describe a heuristic search algorithm for Markov decision problems, called LAO*, that is derived from the classic heuristic search algorithm AO*. LAO* shares the advantage heuristic search has over dynamic programming for simpler classes of problems: it can find optimal solutions without evaluating all problem states. The derivation of LAO* from AO* makes it easier to generalize refinements ...
متن کاملFHHOP: A Factored Hybrid Heuristic Online Planning Algorithm for Large POMDPs
Planning in partially observable Markov decision processes (POMDPs) remains a challenging topic in the artificial intelligence community, in spite of recent impressive progress in approximation techniques. Previous research has indicated that online planning approaches are promising in handling large-scale POMDP domains efficiently as they make decisions “on demand” instead of proactively for t...
متن کامل